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DISPLAYING MULTIPLE SLICE IMAGES 



FIELD OF THE INVENTION 

This invention relates generally to medical imaging, and more particularly to 
5 displaying multiple slice images. 

COPYRIGHT NOTICE/PERMISSION 

A portion of the disclosure of this patent document contains material which is 
subject to copyright protection. The copyright owner has no objection to the facsimile 
10 reproduction by anyone of the patent document or the patent disclosure as it appears in the 
Patent and Trademark Office patent file or records, but otherwise reserves all copyright 
rights whatsoever. The following notice applies to the software and data as described 
below and in the drawings hereto: Copyright © 1999, TeraRecon, Inc., All Rights 
Reserved. 

15 

BACKGROUND OF THE INVENTION 

An X-ray computerized axial tomography (CT)apparatus can be used to visualize 
the position of a biopsy needle during a biopsy procedure on a subject. A continuously 
scanning X-ray CT apparatus has been used to observe the motion of a biopsy needle 
20 during a biopsy in real time. It is reported that it is difficult to accurately understand the 
position of the biopsy needle when guided by X-ray CT having a time series images of a 
single cross section of the subject. 

It is desirable to display the time series images of two or more cross-sections 
simultaneously for biopsy needle localization. Similarly, in interventional radiology, it is 
25 necessary to operate scalpels, needles, or catheters dynamically. In order to respond to the 
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dynamic operation of scalpels, needles, and catheters, it is desirable to display time series 
images of two or more cross-sections simultaneously. 

X-ray CT scanners that acquire data of two or more cross-sections using a multi- 
line X-ray detector are able to display the image of two or more cross-sections 
simultaneously by reconstructing the projection data of two or more cross-sections 
acquired by the multi-line X-ray detector. The reconstructed images are conventionally 
displayed as shown in Figures 4 and 5 (prior art). The display formats illustrated in 
Figures 4 and 5 are explained with reference to Figure 2, which illustrated the spatial 
relationship between a subject, a region of interest, a biopsy needle inserted in a subject, 
and CT slices from a four-line X-ray detector. 

Figure 4 illustrates a display format in which images acquired with the four-line X- 
ray detector are displayed in a two-by-two format on the display area of a single display 
screen 33. The reconstructed image of slice- 1 102 in Figure 2 is displayed on image 
display area 111. A "1" appears as slice number 1 19 in the upper left corner of the image 
display area 111. The reconstructed image of slice-2 103 in Figure 2 is displayed on image 
display area 1 12 in the upper right corner of the display area as slice number 2. The 
reconstructed image of slice-3 104 in Figure 2 is displayed on image display area 113 in 
the lower left corner of the display area as slice number 3. The reconstructed image of 
slice-4 105 in Figure 2 is displayed on image display area 1 14 in the lower right corner of 
the display area as slice number 4. Because the images from the four-line X-ray detectors 
are displayed in two rows and two columns on one display screen, it is difficult to grasp 
the spatial relationship and continuity of the body axis direction of the subject. 

Figure 5 illustrates a display format in which images acquired with four-line X-ray 
detector are displayed in a two by one format on the display area of two display screens 34 
and 35. The reconstructed image of cross section-1 102 in Figure 2 is displayed on image 
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display area 11 1 in a display screen 34. On the left side of the display area, the slice 
number is displayed as 1. The reconstructed image of cross section-2 103 is displayed on 
image display area 112 in a display screen 34. On the right side of the display area, the 
slice is displayed as 2. The reconstructed image of cross section-3 104 is displayed on 
5 image display area 1 13 in a display screen 35. On the left side of the display area, the slice 
is displayed as 3. The reconstructed image of cross section-4 105 is displayed on image 
display area 1 14 in a display screen 35. On the right side of the display area, the slice is 
displayed as 4. Because the images of each cross section from the four-line X-ray detector 
are displayed on two columns and one row on two display screens, it is difficult to grasp 
10 spatial relation and continuity of the body axis direction of a subject. 

Therefore, it is desirable to provide an image display apparatus that facilitates an 
accurate understanding of the position of a medical instrument from the displayed images 
of two or more cross sections of a subject during an invasive procedure. 

1 5 SUMMARY OF THE INVENTION 

A time series of multiple cross-sectional images of a subject are displayed in 
unique display formats synchronized with the acquisition of the images to provide a 
precise location for an invasive medical instrument, thus enabling accurate monitoring of 
the state and motion of the instrument during a procedure. The images are acquired 

20 through real time data acquisition apparatus, such as a real time X-ray CT scanner with a 
multi-line X-ray detector. Each image is displayed in a display area that is deformed to 
provide depth perception. Multiple display areas are displayed simultaneously on a single 
image display unit and the display areas can be adjusted to provide easy and continuous 
comparison of the spatial relationships among the images. In another aspect of the 

25 invention, the display areas are overlapped to provide additional depth perception. In yet 
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another aspect of the invention, each display area is assigned an opacity so that one or 
more display areas can been seen behind an adjacent display area when overlapped. In 
still a further aspect of the invention, the each display area is assigned an opacity and 
displayed on a three-dimensional image reconstructed with previously acquired data. 
5 Thus, the invention enables easy comparison among a time series of adjacent 

cross-sections of a subject, and of spatial information of regions of interest in the images, 
improving the safety and simplicity of invasive procedures on a subject, such as biopsy 
techniques and interventional radiology that are performed under X-ray CT control. 

In addition to the aspects and advantages of the present invention described in this 
10 summary, further aspects and advantages of the invention will become apparent by 
reference to the drawings and by reading the detailed description that follows. 



BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a schematic block diagram showing a configuration of an image display 
15 apparatus of an X-ray CT scanner according to one embodiment of the invention. 

Figure 2 is a figure illustrating a spatial relationship between a subject, a region of 
interest, a biopsy needle, and CT slices. 

Figure 3 is a figure showing a spatial relationship and temporal relationship 
between a subject, a region of interest, a biopsy needle, and CT slices. 
20 Figure 4 is a figure showing a prior art method to display four images of CT slices 

on one display screen. 

Figure 5 is a figure showing a prior art method to display four images of CT slices 
on two display screens. 

Figure 6 is a figure showing a prior art method to display four images of CT slices 
25 on one display screen. 
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Figures 7-20 are exemplary display formats of four images of CT slices shown on 
one display screen by the embodiment of the invention illustrated in Figure 1. 

Figure 21 is a block diagram of a data-processing unit suitable for use with the 
invention. 

DETAILED DESCRIPTION OF THE INVENTION 

In the following detailed description of embodiments of the invention, reference is 
made to the accompanying drawings in which like references indicate similar elements, 
and in which is shown by way of illustration specific embodiments in which the invention 
may be practiced. These embodiments are described in sufficient detail to enable those 
skilled in the art to practice the invention, and it is to be understood that other 
embodiments may be utilized and that logical, mechanical, electrical, functional and other 
changes may be made without departing from the scope of the present invention. The 
following detailed description is, therefore, not to be taken in a limiting sense, and the 
scope of the present invention is defined only by the appended claims. 

Figure 1 is a block diagram showing one embodiment of the present invention. A 
data acquisition apparatus 10 collects projection data of a subject by electro-magnetic 
radiation from the circumference and measures the transmitted dose. The data acquisition 
apparatus 10 is described herein as an X-ray computerized axial tomography (CT) scanner, 
such as an electron beam scanning type X-ray CT scanner, for purposes of explanation but 
the invention functions similarly in other apparatus that produce temporal images in two or 
more planes, such as a magnetic resonance (MR) or ultrasound apparatus, and is not 
limited to use with X-ray CT scanners. 

The apparatus 10 controls an electron beam 13 emitted from an electron gun 12 for 
scanning on an X-ray target 11 annul arly located around a subject. The X-ray beam 
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generated by the X-ray target 11 transmits the cross section of a subject on a table 16, and 
a multi-line X-ray detector 14 intercepts the transmitted X-ray beam. An X-ray CT 
scanner that uses a rotating gantry equipped a rotating-anode X-ray tube and a multi-line 
X-ray detector is also contemplated as within the scope of the invention. A four-line X- 
5 ray detector is used for explanation of the multi-line X-ray detector 14 but the invention 
can be practiced other X-ray detectors, such as an area X-ray detector, and the invention is 
not limited by the examples herein. 

A data acquisition circuit 15 converts the output current of the multi-line X-ray 
detector 14 into digital data. By using the multi-line X-ray detector 14, the apparatus 

10 collects data of multiple cross-sections of the subject simultaneously. A reconstruction- 
processing unit 20 performs pre-processing, reconstruction processing, and post- 
processing of the acquired data, and creates images of multiple cross sections of the 
subject simultaneously within a time synchronized with data acquisition. 

An image display apparatus 30 has an image display section 31 that has display 

15 area 131, display area 132, display area 133 and display area 134 that display the temporal 
images of four cross-sections simultaneously acquired with the multi-line X-ray detector 
14. The image display apparatus 30 has a display parameter control panel 32 to control 
the displayed images in the display areas. The control panel 32 has x-direction display 
control boxes 135 for controlling the displayed images in a first direction and a z-direction 

20 display control box 136 for controlling the displayed images in a second direction 

orthogonal to the first direction. It will be apparent that there are as many x-direction 
control boxesl35 as there are displayed images. In one embodiment, the control boxes 
135, 136 are implemented as knobs or dial. In this embodiment, the control boxes 135 
appears as four knobs or dials, one for each displayed image. 
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The control panel 32 allows the observer to deform the display format of each 
display area 131-134 to provide depth perception, and to change the display format for 
easy comparison of adjacent slices. Conventional texture mapping techniques can be 
employed to create the deformed slices. Frame coordinates are calculated and the 
resulting frame is drawn to enclose the slices (deformed or non-deformed) to create each 
display area 131-134. 

The control panel 32 also allows the observer to overlap adjacent display areas and 
to give a different opacity to each display area. Each display area with a different opacity 
can then be arranged on a three-dimensional image reconstructed with previously acquired 
data. In one embodiment, the opacity for a displayed image is input as a numerical value, 
e.g., 0 - 100%. In another embodiment, a slider bar for each displayed image is used to 
input the opacity. Similarly, in one embodiment, a slider bar is also used to control 
magnification of each displayed image, which results in the overlapping of adjacent 
display areas. 

Figure 2 illustrates spatial relationship between a subject 101, a region of interest 
106, a biopsy needle inserted in a subject 107, and CT slices 102-105 from a four-line X- 
ray detector. The subject 101 is a patient lying on the table 16 in Figure 1. The X-ray 
beam generated by the X-ray target transmits the cross section of the subject 101, and the 
four-line X-ray detector 14 intercepts the transmitted X-ray beam. The data acquisition 
circuit 15 converts output current of the four-line X-ray detector into digital data. 

A top view 38 of the subject 101 as projected on an x-z plane and a side view 39 of 
the subject 101 as projected on an x-y plane are shown in Figure 2. The x axis is the 
direction from upper left corner to upper right corner in a plane parallel to a cross-section, 
the y axis is the direction from the upper left corner to the lower left corner, and the z axis 
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is the direction from the foot to the head of patient that intersects perpendicularly with the 
x-y plane. 

In the top view 38 of Figure 2, slice-1 102, slice-2 103, slice-3 104 and slice-4 105 
are slices reconstructed using data detected with each detector line of the four-line X-ray 
5 detector 14. It shows a region of interest 106 in the slice 104 and slice 105, a biopsy 

needle 107 in the slice 102, 103, 104 and 105, and x-z coordinates 108. The side view 39 
of Figure 2 shows the region of interest 106, the biopsy needle 107, and x-y coordinates 
109. 

Figure 3 shows reconstructed images of slice-1 in column 111, reconstructed 

10 images of slice-2 in column 1 12, reconstructed images of slice-3 in column 1 13, and 
reconstructed images of slice-4 in column 1 14, each image reconstructed using the 
projection data in the slice-1 102, slice-2 103, slice-3 104, slice-4 105 in Figure 2 detected 
with each detector line of four-line X-ray detector. It shows the reconstructed images at 
time-1 in row 115, reconstructed images at time-2 in row 116, reconstructed images at 

15 time-3 in row 117, and reconstructed images at time-4 in row 1 18, each image 

reconstructed using the projection data at time-1, time-2, time-3, and time-4 detected with 
each detector line of four-line X-ray detector. 

On the upper left corner of the display area of each cross-section image, a slice 
number 119 is displayed. The cross section 120 shown in each cross-section image is the 

20 cross section of the subject 101. The region of interest 121 shown in cross section 113 and 
cross section 114 is the cross section of the region of interest 106. 

The biopsy needle 122 in the slice-1 1 1 1 at time-1 1 15, at time-2 1 16, at time-3 
117, and time-4 118 shows the biopsy needle 107 contained in the slice-1 102. The biopsy 
needle 123 in the slice-2 112 at time-2 116, at time-3 117, and time-4 118, shows the 

25 biopsy needle 107 contained in the slice 103. The biopsy needle 124 in the slice-3 1 13 at 
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time-3 1 17, and time-4 118 shows the biopsy needle 107 contained in the slice 104. The 
biopsy needle 125 in the slice-4 1 14 at time-4 118 shows biopsy needle 107 contained in 
the slice 104. Figures 7-20 illustrate various embodiments of the invention in displaying 
the slices at time-4 118. 

5 As described previously, Figure 4 and Figure 5 show conventional prior art display 

formats. In the prior display format of Figure 4, the images of the cross sections from a 
four-line X-ray detector are displayed in two rows and two columns on one display screen, 
making it difficult to grasp the spatial relationship and continuity of the body axis 
direction of the subject. In the prior art display format of Figure 5, the images of the cross 

10 section from the four-line X-ray detector are displayed in two columns and one row on 
two display screens, also making it difficult to grasp the spatial relation and continuity of 
the body axis direction of a subject. 

Figure 6 illustrates a prior art display format designed to alleviate the problems of 
the display formats of Figure 4 and Figure 5. In the display format of Figure 6, the width 

15 (x-direction) of each display area is shortened, while the height (y-direction) of each 

display area is maintained. The reconstructed image of cross section- 1 102 is displayed on 
the image display area 126 of a display screen 36. On the left corner of the display area, a 
slice number 119 is displayed as 1. The reconstructed image of cross section-2 103 is 
displayed on image display area 127 in the display screen 36. The reconstructed image of 

20 cross section-3 104 is displayed on image display area 128 in the display screen 36. The 
reconstructed image of cross section-4 105 is displayed on image display area 129 in the 
display screen 36. The reconstructed images of four cross sections can now be 
horizontally displayed side by side on one display screen, making the comparison of the 
four cross sections easier than in the display formats of Figure 4 or Figure 5. Additionally, 

25 the distance between regions of interest in two adjacent display areas is shorter than the 
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corresponding in Figure 5, so viewing the cross sections during the invasive operation is 
easier. However, because the display format in Figure 6 provides no information 
regarding the relationship and order among the images, this prior art display format does 
not enable easy understanding of the spatial relation and continuity of the body axis 
5 direction of a subject. 

Figures 7-20 are examples of display formats created by the image display 
apparatus 30 of the present invention. The image display apparatus 30 displays multiple 
images side-by-side on a single display screen, and provides information and control over 
the x and y directions of the images. As in the prior art display format of Figure 6, the 

10 width (x-direction) of each display area is shortened, while the height (y-direction) of each 
display area is maintained so that the display aspect ratio of image is changed. Unlike the 
display formats of Figures 4, 5 and 6, there is an individual x-direction display control for 
each display area and a global z-direction display control for all the display areas. The 
current directions for the x and y axes are indicated by arrows as is further described in 

15 conjunction with each of the Figures 7-20. Thus, as compared with the display formats of 
Figures 4, 5 and 6, the arrows enable the observer to easily understand the x-direction of 
the images and understand the order and relation in the z-direction of multiple images. 
Furthermore, changing the directions of the x and y axes cause the display areas to change 
accordingly to provide greater depth perception and change the displayed relationship 

20 among the images. 

Figure 7 illustrates three exemplary display formats 41, 42 and 43. In each case, 
the reconstructed image of cross section- 1 102 is displayed on the image display area 131 
of the display screen 37. In the left corner of the display area 13 1, the slice number 1 19 is 
displayed as 1. The reconstructed image of cross section-2 103 is displayed on the image 

25 display area 132, screen 37. In the left corner of the display area 132, the slice number 1 19 



00509 1.P002 



-11- 



is displayed as 2. The reconstructed image of cross section-3 104 is displayed on the 
image display area 133 of the display screen 37. In the left corner of the display area 133, 
the slice number 119 is displayed as 3. The reconstructed image of cross section-4 105 is 
displayed on the image display area 134 of the display screen 37. In the left corner of the 
5 display area 134, the slice number 1 19 is displayed as 4. 

Additionally, each image display area 131-134 has an x-direction display control 
box 135 that indicates the x-direction of the image and controls characteristics of the 
display such as inclination of the x-direction. For all four image display areas 131-134, 
there is one z-direction display control box 136 that indicates the order of images in the z- 
10 direction and controls order of images in the z-direction and arranges images in the z- 
direction. 

In display format 41, each x-direction display control box 135 is set to the right, 
and the z-direction display control box 136 is set to the right. In display format 42, each x- 
direction display control box 135 is tilted to the lower right direction to deform the image 

15 display area and to give depth perception. It is sufficient to only to deform the shape of 
the frame of the image display area in display format 42, and it is not necessary to deform 
image itself. In display format 43, each x-direction display control box 135 is set to the 
lower left direction to deform the image display area and to give depth perception. 

Figure 8 illustrates three exemplary display formats 44, 45 and 46. In display 

20 format 44, each x-direction display control box 135 is set to the right, and z-direction 
display control box 136 is set to the left. The order of display area in the z-direction of 
multiple images can be changed by operation of z-direction display control box 136. In 
display format 45, each x-direction display control box 135 is set to the upper right 
direction to deform image display area and to give depth perception. It is sufficient only to 

25 deform the shape of frame of the image display area in display format 45, and it is not 
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necessary to deform image itself. In display format 46, each x-direction display control 
box 135 is set to the upper left direction to deform image display area and to give depth 
perception. 

Figure 9 illustrates two additional exemplary display formats 47 and 48. Display 
format 41 in Figure 9 is same as display format 41 in Figure 7 in which each x-direction 
display control box 135 is set to the right, and the z-direction display control box 136 is set 
to the right. In display format 47, the x-direction display control boxes 135 are set to the 
lower left direction in the display area 131, 132 and 133, and x-direction display control 
box 135 is set to lower right direction in the display area 134 to deform image display area 
and to give depth perception. Distance between the region of interest displayed on the 
image display area 133 and the image display area 134 becomes shorter than in Figure 7. 
It is sufficient to only deform the shape of frame of the image display area in display 
format 47, and it is not necessary to deform image itself. Thus, the observer can observe 
the cross-sections as if he actually cut the subject between slice-3 and slice-4 and folded 
the slices open as if they as if they were pages in a book. In display format 48, the x- 
direction display control boxes 135 in the display area 131 and 132 are set to the lower left 
direction, and the x-direction display control boxes 135 in the display area 133 and 134 are 
set to lower right direction to deform image display areas and to give depth perception. 
Thus, the observer can observe the cross-sections as if he actually cut in the subject 
between slice-2 and slice-3 and folded the slices open as if they as if they were pages in a 
book. 

Figure 10 illustrates two additional exemplary display formats 49 and 50. Display 
format 44 in Figure 10 is same as display format 44 in Figure 8 in which each x-direction 
display control box 135 is set to the right, and the z-direction display control box 136 is set 
to the left. In display format 49, the x-direction display control boxes 135 in the display 
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area 131, 132 and 133 is set to the upper right direction, and the x-direction display control 
box 135 in the display area 134 is set to upper left direction to deform the image display 
area and give depth perception. The distance between the region of interest displayed on 
the image display area 133 and the image display area 134 becomes shorter than in Figure 
5 8. It is sufficient to only deform the shape of frame of the image display area in display 
format 49, and it is not necessary to deform the image itself. Thus, the observer can 
observe the cross-sections as if he actually cut the subject between slice-3 and slice-4 and 
folded the slices open as if they as if they were pages in a book. In display format 50, the 
x-direction display control box 135 in the display area 131 and 132 is set to the upper right 

10 direction, and the x-direction display control box 135 in the display area 133 and 134 are 
set to upper left direction to deform image display areas and give depth perception. Thus, 
the observer can observe the cross-sections as if he actually cut the subject between slice-2 
and slice-3 and folded the slices open as if they as if they were pages in a book. 

Figure 11 and Figure 12 illustrate display formats in which width of the display 

15 area is made narrower than in Figure 7 and Figure 8. By changing the width of the display 
area and the inclination in the depth direction, more natural depth perception may be 
obtained. 

Display format 41 in Figure 1 1 corresponds to display format 41 in Figure 7 in 
which each x-direction display control box 135 is set to the right, and the z-direction 

20 display control box 136 is set to the right. Display format 42 in Figure 1 1 corresponds to 
display format 42 in Figure 7 in which each x-direction display control box 135 is set to 
the lower right direction to deform the image display area and to give depth perception, 
and the z-direction display control box 136 is set to the right. In display format 51 in 
Figure 11, each x-direction display control box 135 is set to the lower right direction to 

25 deform the image display area to give depth perception, and the z-direction display control 
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box 136 is set to the right. The length of the z-direction display control box 136 is set 
shorter than in display format 42 to shorten the width of each image display area 137, 138, 
139 and 140 as compared to display areas 131, 132, 133 and 134 in display format 42. It 
is sufficient to only deform the shape of the frames of the image display area and to 
5 change aspect ratio of the image in display format 5 1 , and it is not necessary to deform 
images themselves. Thus, the observer may get a higher depth perception than display 
format 42, and the observer may observe region of interest or the needle in the adjacent 
display areas more precisely than display format 42. 

Display format 44 in Figure 12 corresponds to display format 44 in Figure 8 in 

10 which each x-direction display control box 135 is set to the right, and the z-direction 

display control box 136 is set to the left. Display format 45 in Figure 12 corresponds to 
display format 45 in Figure 8 in which each x-direction display control box 135 is set to 
the upper right direction to deform the image display area and to give depth perception, 
and the z-direction display control box 136 is set to the left. In display format 52 in Figure 

15 12, each x-direction display control box 135 is set to the upper right direction to deform 
the image display area to give depth perception, and the z-direction display control box 
136 is set to the left. The length of the z-direction display control box 136 is set shorter 
than in display format 45 to shorten the width of each image display area 137, 138, 139 
and 140 compared to display areas 131, 132, 133 and 134 of display format 45. It is 

20 sufficient to only deform the shape of frame of the image display area and to change the 
aspect ratio of the image in display format 52, and it is not necessary to deform the image 
itself. Thus, the observer may get higher depth perception than the example of display 
format 45, and the observer may observe the region of interest or the needle in an adjacent 
display area more precisely than the example of display format 45. 
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Figure 13 and Figure 14 illustrate display formats in which image display areas are 
deformed into parallelograms. Display format 41 in Figure 13 is identical to display 
format 41 in Figure 7 in which each x-direction display control box 135 is set to the right, 
and the z-direction display control box 136 is set to the right. Display format 53 in Figure 
5 13 is similar to display format 42 in Figure 7 in which each x-direction display control box 
135 is set to the lower right direction to deform image display area to give depth 
perception, and the z-direction display control box 136 is set to the right but the shape of 
the display area is different than in display format 42. In display area 53, the image 
display area 141 for slice-1, image display area 142 for slice-2, image display area 143 for 

10 slice-3, and image display area 144 for slice-4 are deformed into parallelograms. It is 

sufficient only to deform the shape of the frame of the image display area in display area 
53, and it is not necessary to deform image itself. Display format 54 in Figure 13 is 
similar to display format 43 in Figure 7 in which each x-direction display control box 135 
is set to the lower left to deform the image display area to give depth perception, and the z- 

15 direction display control box 136 is set to the right but the shape of the display area is 
different than in display format 43. 

Display format 44 in Figure 14 is identical display format 44 in Figure 8 in which 
each x-direction display control box 135 is set to the right, and the z-direction display 
control box 136 is set to the left. Display format 55 in Figure 14 is similar to display 

20 format 45 in Figure 8 in which each x-direction display control box 135 is set to the upper 
right direction to deform image display area and to give depth perception, and the z- 
direction display control box 136 is set to the left but the shape of the display area is 
different than in display format 45. In display format 55, image display area 141 for slice- 
1, image display area 142 for slice-2, image display area 143 for slice-3, and image 

25 display area 144 for slice-4 are deformed into parallelograms. It is sufficient to only 
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deform the shape of the frame of the image display area in display format 55, and it is not 
necessary to deform image itself. Display format 56 in Figure 14 is similar to display 
format 46 in Figure 8 in which each x-direction display control box 135 is set to the upper 
left direction to deform image display area to give depth perception, and the z-direction 
5 display control box 136 is set to the left but the shape of the display area is different than 
in display format 46 by having the frame deformed into a parallelogram. 

In the display formats 57 and 58 illustrated in Figure 15 and Figure 16, only a 
narrow part of image is shown without displaying all areas of image. Display format 44 in 
Figure 15 corresponds to display format 44 in Figure 8 in which each x-direction display 

10 control box 135 is set to the right, and the z-direction display control box 136 is set to the 
left. Display format 45 in Figure 15 corresponds to display format 45 in Figure 8 in which 
each x-direction display control box 135 is set to the upper right direction to deform image 
display area and to give depth perception, and the z-direction display control box 136 is 
set to the left. In display format 57 in Figure 15, the width of images of slice-1, slice-2, 

15 slice-3, and slice-4 is enlarged compared to display format 45 in Figure 15, and displayed 
image area 145, 146, 147, and 148 have larger width than image display areas 131, 132, 
133, and 134 in display format 45 of Figure 15. The center of magnification and the 
magnification ratio can be set with the x-direction display control box 135. It is sufficient 
to only deform the shape of frame of the image display area and to change aspect ratio of 

20 the image in display format 57, and it is not necessary to deform image itself. 

Display format 44 in Figure 16 corresponds to display format 44 in Figure 8 in 
which each x-direction display control box 135 is set to the right, and the z-direction 
display control box 136 is set to the left. Display format 46 in Figure 16 corresponds to 
display format 46 in Figure 8 in which each x-direction display control box 135 is set to 

25 the upper left direction to deform image display area to give depth perception, and the z- 
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direction display control box 136 is set to the left. In display format 58 in Figure 16, the 
width of images of slice-1, slice-2, slice-3, and slice-4 is enlarged compared to 46 in 
Figure 16, and displayed image area 145, 146, 147, and 148 have larger width than image 
display areasBl, 132, 133, and 134 in display format 46 of Figure 16. Center of 
5 magnification and magnification ratio can be set with the x-direction display control box 
135. It is sufficient to only deform the shape of frame of the image display area and to 
change aspect ratio of the image in display format 58, and it is not necessary to deform 
image itself. 

Figure 17 and Figure 18 illustrate display formats 59 and 60, respectively, in which 

10 images are arranged in an overlapping fashion. Each image is assigned an opacity and if 
the opacity of an image is less than a threshold value, images it overlaps are shown. 
Display format 44 in Figure 17 is identical to display format 44 in Figure 8 in which each 
x-direction display control box 135 is set to the right, and the z-direction display control 
box 136 is set to the left. Display format 45 in Figure 17 corresponds to display format 45 

15 in Figure 8 in which each x-direction display control box 135 is set to the upper right 
direction to deform image display area and to give depth perception, and the z-direction 
display control box 136 is set to the left. In display format 59 of Figure 17, the image 
width of slice-1, slice-2, slice-3, and slice-4 and image display area 149, 150, 151, and 152 
have larger width than the image width of slice-1, slice-2, slice-3, and slice-4 and image 

20 display area 131, 132, 133, and 134 in display format 45 of Figure 8. It is sufficient to only 
deform the shape of frame of the image display area and to change aspect ratio of the 
image in display format 59, and it is not necessary to deform image itself. Additionally, 
the images of slice-1, slice-2, and slice-3 and image display area 149, 150, and 151 have 
an opacity less than the threshold so that images and image display areas behind them can 

25 be seen (shown in phantom in Figure 17). 
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Display format 41 in Figure 18 corresponds to display format 41 in Figure 7 in 
which each x-direction display control box 135 is set to the right, and the z-direction 
display control box 136 is set to the right. Display format 43 in Figure 18 corresponds to 
display format 43 in Figure 7 in which each x-direction display control box 135 is set to 
5 the lower left direction to deform image display area to give depth perception, and the z- 
direction display control box 136 is set to the right. Display format 60 in Figure 18 is an 
example in which the image width of slice- 1, slice-2, slice-3, and slice-4 and image 
display area 149, 150, 151, and 152 have larger width than the image width of slice- 1, 
slice-2, slice-3, and slice-4 and image display area 131, 132, 133, and 134 in display 

10 format 43 of Figure 1 8. It is sufficient to only deform the shape of frame of the image 

display area and to change the aspect ratio of the image in display format 60, and it is not 
necessary to deform image itself. Images of slice-2, slice-3, and slice-4 and image display 
areas 150, 151, and 152 have opacity less than the threshold so that images and image 
display areas behind them can be seen (shown in phantom in Figure 18). 

15 Figure 19 illustrates display formats 61 and 62 showing the overlapping of image 

display areas for transparent images. An image group 153 is a group of images of slices 
projected on the plane defined by the biopsy needle 107 and y-axis as illustrated in Figure 
2. Image 155 is the projected image of slice 102, image 156 is the projected image of slice 
103, image 157 is the projected image of slice 104, and image 158 is the projected image 

20 of slice 105. An image group 154 a group of images of slices projected on the plane that 
intersects perpendicularly with the plane defined by the biopsy needle 107 and y-axis and 
includes y-axis in Figure 2. Image 159 is the projected image of slice 102, image 160 is 
the projected image of slice 103, image 161 is the projected image of slice 104, and image 
162 is the projected image of slice 105. By projecting images on two planes that intersect 

25 perpendicularly, the motion of a biopsy needle may be observed more accurately. On the 
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image, a guideline 171, 172 is displayed between the insertion point of a biopsy needle on 
the surface of a subject and the region of interest, and operation of the biopsy needle is 
made easy. The x-direction display control box 135, and z-direction display control box 
136, and the display direction of images can be set up initially as shown in display format 
61. As shown in display format 62, changing the x-direction display control box 135 and 
the z-direction display control box 136 changes the display direction. 

Figure 20 illustrates display formats 63 and 64 in which image display areas are 
overlapped and displayed on a three-dimensional image with a different opacity. In 
display format 63 and display format 64, an image group 163 is group of images of slices 
projected on the plane defined by the biopsy needle 107 and y-axis as illustrated in Figure 
2. Image 165 is the projected image of slice 102, image 166 is the projected image of slice 
103, image 167 is the projected image of slice 104, and image 168 is the projected image 
of slice 105. A three-dimensional image 164 is a three-dimensional image created by the 
CT scan preceding the insertion of biopsy needle and displayed with the same coordinate 
system as image groupl63. In this example, image 165 of slice-1, image 166 of slice-2, 
image 167 of slice-3, and image 168 of slice-4 are overlapped and displayed on the three- 
dimensional image 164 that has same coordinate system with slices. By adjusting the 
display opacity of the three-dimensional image 164, the display formats 63 and 64 can 
show the three-dimensional image and the images of slice-1, slice-2, slice-3, and slice-4 as 
different opacities, making them easy to distinguish. In display format 63, subtraction 
images of slice-1, slice-2, slice-3, and slice-4 can be displayed as image 165 of slice-1, 
image 166 of slice-2, image 167 of slice-3, and image 168 of slice-4 so that only biopsy 
needle can be displayed on the three-dimensional image 164. Since a biopsy needle has a 
specific CT value, the invention extracts only the portion of a biopsy needle in each image 
and displays it on the three-dimensional image 164 so that only the biopsy needle is seen. 
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By displaying on the image a guideline 173 that connects the point of insertion of the 
needle on the surface of a subject and the region of interest, operation of a biopsy needle 
can be made easy. Display format 64 in Figure 20 illustrate the change in display direction 
caused by changing the x-direction display control box 135 and the z-direction display 
control box 136. 

Turning now to Figure 21, one embodiment of a computer system 400 for use with 
the present invention is described. The system 400, includes a processor 450, memory 455 
and input/output capability 460 coupled to a system bus 465. The memory 455 is 
configured to store instructions which, when executed by the processor 450, perform the 
functions of the invention described herein. The memory 455 may also store the various 
tables previously described and the results of the processing of the data within those 
tables. Input/output 460 provides for the delivery and display of the images or portions or 
representations thereof. Input/output 460 also provides for access to the image data 
provided by other components and for user control of the operation of the invention. 
Further, input/output 460 encompasses various types of computer-readable media, 
including any type of storage device that is accessible by the processor 450. One of skill 
in the art will immediately recognize that the term "computer-readable medium/media" 
further encompasses a carrier wave that encodes a data signal. 

The instructions may be written in a computer programming language or may be 
embodied in firmware logic. If written in a programming language conforming to a 
recognized standard, such instructions can be executed on a variety of hardware platforms 
and for interface to a variety of operating systems. In addition, the present invention is not 
described with reference to any particular programming language. It will be appreciated 
that a variety of programming languages may be used to implement the teachings of the 
invention as described herein. Furthermore, it is common in the art to speak of software, 
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in one form or another (e.g., program, procedure, process, application, module, logic...), as 
taking an action or causing a result. Such expressions are merely a shorthand way of 
saying that execution of the software by a computer causes the processor of the computer 
to perform an action or a produce a result. 
5 The foregoing description of Figure 4 is intended to provide an overview of 

computer hardware and other operating components suitable for implementing the 
invention, but is not intended to limit the applicable environments. It will be appreciated 
that the computer system 440 is one example of many possible computer systems which 
have different architectures. A typical computer system will usually include at least a 

10 processor, memory, and a bus coupling the memory to the processor. One of skill in the 
art will immediately appreciate that the invention can be practiced with other computer 
system configurations, including multiprocessor systems, minicomputers, mainframe 
computers, and the like. The invention can also be practiced in distributed computing 
environments where tasks are performed by remote processing devices that are linked 

15 through a communications network. 
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